High Efficiency Video Coding (HEVC) is the latest international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC) group of video coding experts from ITU-T Study Group. The HEVC standard relies on a block-based coding structure which divides a picture into multiple non-overlapped square Coding Tree Units (CTUs). Each CTU consists of multiple Coding Tree Blocks (CTBs) and each CTB is for one color component. FIG. 1 illustrates an exemplary system block diagram of a Video Encoder 100 based on the HEVC standard. Intra Prediction 110 provides intra predictors based on reconstructed blocks of a current picture, and Inter Prediction 112 performs motion estimation (ME) and motion compensation (MC) to provide inter predictors based on video data from other picture or pictures. Switch 114 selects an intra predictor from Intra Prediction 110 or an interpredictor from Inter Prediction 112 for each block in the current picture. The selected predictor is supplied to Adder 116 to be subtracted from the input video data in order to form prediction errors, also called residues. The prediction errors are then processed by Transformation (T) 118 followed by Quantization (Q) 120. The transformed and quantized residues are then coded by Entropy Encoder 134 to form a video bitstream corresponding to the compressed video data. The video bitstream associated with the transform coefficients is then packed with side information. The side information may also be subject to entropy coding to reduce required bandwidth. Accordingly, the data associated with the side information are provided to Entropy Encoder 134 as shown in FIG. 1. When an interprediction mode is used, a reference picture or pictures have to be reconstructed at the encoder end as well. Consequently, the transformed and quantized residues are processed by Inverse Quantization (IQ) 122 and Inverse Transformation (IT) 124 to recover the residues. The residues are then added back to the selected predictors at Reconstruction (REC) 126 to produce reconstructed blocks. The reconstructed blocks may be stored in Reference Picture Buffer 132 and used for prediction of other pictures.
As shown in FIG. 1, incoming video data undergoes a series of processing in the video encoding system. The reconstructed blocks generated from REC 126 may be subject to various impairments due to a series of encoding processing. Accordingly, various in-loop processing is applied to the reconstructed blocks before the reconstructed blocks are stored in the Reference Picture Buffer 132 in order to improve video quality. In the HEVC standard, Deblocking Filter (DF) 128 and Sample Adaptive Offset (SAO) 130 are applied to the reconstructed blocks before storing in the Reference Picture Buffer 132 to enhance picture quality. The in-loop filter information may have to be incorporated in the video bitstream so that a decoder can properly recover the required information. For example, SAO information from SAO 130 is provided to Entropy Encoder 134 for incorporation into the video bitstream.
In the HEVC standard, SAO processing is utilized to reduce the distortion in the reconstructed blocks. FIG. 2 illustrates a system block diagram of an exemplary HEVC-based Video Decoder 200 including Deblocking Filter (DF) 224 and Sample Adaptive Offset (SAO) 226. Entropy Decoder 210 is used to parse and recover the coded syntax elements related to residues, motion information and other control data. Switch 216 selects Intra Prediction 212 or Inter Prediction 214 according to decoded mode information, and the selected predictors are supplied to reconstruction (REC) 218 to be combined with the recovered residues. Besides performing entropy decoding on compressed video data, Entropy Decoder 210 is also responsible for entropy decoding of side information and provides the side information to respective blocks. For example, intra mode information is provided to Intra Prediction 212, inter mode information is provided to Inter Prediction 214, sample adaptive offset information is provided to SAO 226 and residues are provided to Inverse Quantization (IQ) 220. The residues are processed by IQ 220, Inverse Transformation (IT) 222 and subsequent reconstruction process to produce reconstructed blocks. The reconstructed blocks are further processed by DF224 and SAO 226 to generate final decoded video. If the currently decoded picture is a reference picture, the final decoded video of the currently decoded picture is also stored in Ref. Pict. Buffer 228 for later pictures in decoding order.
The concept of SAO is to classify the reconstructed pixels in a reconstructed block into categories according to pixel intensities in the reconstructed block. Each category is then assigned an offset coded in the video bitstream and the distortion of the reconstructed block is reduced by adding the offset to the reconstructed pixels in each category. In the HEVC standard, the SAO tool supports two kinds of pixel classification methods: band offset (BO) and edge offset (EO). For each color component (luminance or chrominance), the SAO algorithm divides a picture into non-overlapped regions, and each region selects one SAO type among BO, four EO types, and no process (OFF). The SAO partitioning in the HEVC standard is aligned with the coding tree block (CTB) boundaries to facilitate the CTB-based processing.
For BO, the reconstructed pixels in a region are classified into bands by quantizing the pixel intensities, and the pixel intensity range from a minimum value to a maximum value is equally divided into 32 bands as shown in FIG. 3. Four consecutive bands are subjective to band offset processing, and an offset is derived for each band of the four consecutive bands to reduce the distortion of the reconstructed pixels in the four consecutive bands. The four consecutive bands are located by a syntax element sao_band_position which indicates the starting band of the four consecutive bands. An exemplary four-band group 34 is illustrated in FIG. 3, and the starting band position of this four-band group 34 is indicated by an arrow 32 derived from the syntax element sao_band_position. Four offsets identified by the starting band position are coded into the video bitstream.
For EO, the reconstructed pixels in a region are classified into categories by comparing a current pixel with its neighboring pixels along the direction identified by the EO type as shown in FIG. 4. Table 1 lists the decision for the EO pixel classification according to the HEVC standard, where “C” denotes a current pixel to be classified. The four EO types with selections of neighboring pixels for different orientations are also shown in FIG. 4. The four EO types corresponding to orientations 0°, 90°, 135°, and 45°, where “C1” and “C−1” are the neighboring pixels corresponding to a given EO type as shown in FIG. 4. An offset is derived for all pixels in each category. Four offsets corresponding to category indices 1 through 4 respectively, are coded for one coding tree block (CTB) in the HEVC standard, where reconstructed pixels belong to category index 0 are not compensated by any offset.
TABLE 1CategCondition1C < two neighbors2C < one neighbor && C == one3C > one neighbor && C == one4C > two neighbors0None of the above
SAO information including SAO merging information, SAO type information, and offset information, are interleaved into slice data if SAO is enabled in the current slice. To further reduce side-information, SAO parameters including the SAO type information and offset information of a current CTB can reuse those of its upper or left CTB by using SAO Merge information. Syntax elements for the SAO information consist of sao_merge_left_flag, sao_merge up_flag, sao_type_idx luma, sao type_idx_chroma, sao_eo_class_luma, sao_eo_class_chroma, sao_band_position, sao_offset_abs, and sao_offset_sign. FIG. 5 illustrates merging of SAO regions according to the SAO merging information. In FIG. 5, each block is a region such as a CTB, and the shaded blocks are CTBs after SAO processing. Syntax element sao_merge_left_flag is used to indicate whether a current CTB 52 reuses the SAO parameters of its left CTB 54. The syntax element sao_merge up_flag is used to represents whether the current CTB 52 reuses the SAO parameters of its upper CTB 56. The syntax element sao type_idx represents the selected SAO type (i.e., sao type_idx luma and sao type_idx_chroma for luma component and chroma component respectively). The syntax elements sao_eo_class_luma and sao_eo_class_chroma represent the selected EO type for luma component and chroma component respectively. The syntax element sao_band_position represents the starting band position of the selected bands. SAO processing is applied separately to different color components of the video data, and the color components may correspond to (Y, Cb, Cr), (Y, U, V) or (R, G, B).
A current CTU can reuse SAO parameters of its left neighboring CTU by setting the syntax element sao_merge_left_flag to true, or the current CTU can reuse SAO parameters of its above CTU by setting the syntax element sao_merge up_flag to true, or the current CTU can have new SAO parameters by transmitting a new set of SAO type information and offset information. FIG. 6 illustrates exemplary syntax design for signaling the SAO information for a current block and FIG. 7 illustrates exemplary syntax design for signaling SAO parameters for the current block. In FIG. 6, the SAO merging information is signaled to indicate whether the current block reuses SAO parameters of a neighboring block. If the current block is not merged with any neighboring CTU for SAO processing, EO or BO is selected as the SAO type and four offsets are transmitted for each color component as shown in FIG. 7. The syntax element sao_offset_abs represents the offset magnitude and the syntax element sao_offset_sign represents the offset sign. CIdx indicates a color component index, for example, cIdx=0, 1, 2 correspond to color components Y, Cb, and Cr. The syntax element sao_offset_abs is entropy coded according to the existing HEVC standard using truncated Rice (TR) binarization process. The TR codes comprise a prefix part represented by truncated unary (TU) codes and a remainder part represented by fixed-length codewords without truncation.
While SAO processing in the existing HEVC standard is able to improve performance by adaptively compensate the local intensity offsets, it is desirable to further improve the performance whenever possible in order to achieve an overall efficiency target.